1. Field of the Invention
The present invention relates to a method and apparatus for item recognition and, more specifically, to the automatic recognition of items at a xe2x80x9cpoint of salexe2x80x9d (POS) in a retail outlet, such as at a hardware store checkout.
2. Description of the Related Art
In all retail environments the various items of purchase need to be identified for entry into a computerised inventory and register system. This is very often done by applying a UPC (or similar) barcode to the item and then using the scanner at the POS to scan the barcode in order to produce a unique data entry for passing to the computerised system. This system is popular and effective and used now in all types of retail outlets. However, in the case of some items this approach is simply unrealistic, as applying a bar code to the item can be impractical. For example, in a hardware store a shopper may be purchasing small, loose items such as nuts, bolts and other fasteners. It may not be cost-effective or it may not be possible or convenient to reliably apply a bar code to such items, and the present approach to addressing this problem is to leave the job of item identification to the checkout operators, each checkout being provided with a manual look-up directory whose pages display images of each item sorted by category and presented alongside an individual barcode. The operator will look up the item image in the relevant category and, once it has been identified, will simply scan the accompanying barcode to enter the item into the computerised system. Whilst this system is generally fairly reliable it is naturally time-consuming and inefficient, and relies on an accurate categorisation of the various items and continuous updates to the manual directories. Thought has been given to computerising this procedure, and displaying the directory on a screen for enquiry by the checkout operator, but this approach does little to relieve the basic inefficiency of such an identification procedure at the POS.
Previous attempts have been made at providing a POS checkout system which automates item identification. In U.S. Pat. No. 5,497,314, for an xe2x80x9cAutomated Apparatus and Method for Object Recognition at Checkout Countersxe2x80x9d, an unmanned checkout station is disclosed to which the items to be processed are conveyed by any appropriate means, the items being placed on the conveyor by the customer and automatically conveyed into an enclosed housing containing an illumination source, weight differentiation means and video imaging equipment. Sensors are used to control the operation of the automatic conveyor, and digital image processing techniques are then used to identify the items by taking a digitised image of the object at a known focal point and illumination, and correlating it by way of xe2x80x9ctemplate matchingxe2x80x9d to the set of digitised images contained within the database of the system.
The system of U.S. Pat. No. 5,497,314 uses 3-D images with controlled overhead lighting, and requires a housing to minimise the effect of extraneous light sources. Since the items to be identified are moving on a conveyor belt, the system also requires either a camera lens with auto-focus capability or a location sensor to determine the position of the various items on the belt, to ensure they are in the focal plane of the camera lens when the digitised image is captured. Whilst sophisticated in its complexity the system disclosed is inherently slow and cumbersome and its cost likely to be too high to be competitive in most retail markets.
The present invention aims to address the drawbacks of the prior art, and to that end there is provided a method for recognising an individual item at an identification station from a variety of known products for purposes of entry into an inventory and/or register system, said method comprising the steps of;
providing a viewplate at said identification station;
illuminating said viewplate from the backside to provide a backlit image viewable from the front side of the viewplate;
providing a digital image capturing means positioned relative to the viewplate to capture said backlit image;
capturing and storing a reference image of said viewplate;
supporting the item to be identified adjacent the viewplate;
viewing said item with said digital image capturing means to capture said backlit image;
providing a digitised image signal representing said captured image;
processing said digitised image signal by subtraction of a representation of said stored reference image to remove viewplate artifact, to provide a background-compensated image signal;
image-analysing said background-compensated image signal to generate at least one image descriptor characterising features of size and/or shape of the item to be identified;
providing a computerised product database for storage of data sets representing image descriptors characterising features of size and/or shape for all of said variety of known products;
comparing said at least one item image descriptor with the data stored on said computerised product database to identify data sets with the highest correlation;
displaying a recognition result, being a visual representation of at least one product corresponding to the data sets with the highest correlation; and
providing means for selective verification whereby an operator can confirm the recognition result for entry into the inventory and/or register system.
In a preferred form, the viewplate is arranged for supporting the item to be identified, the backside of the viewplate then being the underside, and the digital image capturing means being arranged above the viewplate to capture the backlit image.
Preferably, said digital image capturing means is a black-and-white digital camera and said digitised image signal is a grey-scale image signal. For an additional level of item discrimination, the digital image capturing means may comprise a colour camera.
In one form of the invention, said identification station is a checkout counter in a retail goods outlet, provided with a display screen for displaying said recognition result.
In a preferred form, said viewplate is associated with a weight data means capturing weight data of said item to be identified, and said weight data is also used in said computerised product database and said comparison step to produce the recognition result.
Preferably, said digitised image signal or said background compensated image signal is subjected to a binary threshold step, and additionally or alternatively said background compensated image signal is subjected to a filtering step to reduce or remove noise and/or errors.
Said image analysing step may include an object segmentation step which serves to isolate a segmented image of an individual item from any other parts of the image. The object segmentation step may involve a selection based on prescribed criteria of size, shape and/or position to select a segmented image of an individual item from other segmented images.
In a preferred embodiment, said generation of said at least one image descriptor involves calculating a representation of at least one parameter of said background-compensated image signal selected from the group of width, length, area, occupation ratio, circularity, perimeter, eccentricity, hole number, hole size, boundary feature, radial diameters signature, minimum radial diameters, maximum radial diameters, mean of radial diameters, standard deviation of radial diameters, radial radii signature, minimum radial radii, maximum radial radii, mean of radial radii, standard deviation of radial radii, first Hu moment, second Hu moment, third Hu moment, fourth Hu moment, fifth Hu moment, sixth Hu moment and seventh Hu moment.
Preferably, said comparison step involves a pattern recognition algorithm, such as a K-Nearest Neighbour technique.
Said recognition result may be a candidate list of a number of said products with the data sets having the highest ranking correlation.
The visual representation may be an image of a product or products stored on the computerised product database, and both the displaying step and the step of selective verification are preferably carried out at the identification station. The display step may include the display of a barcode on said screen for selective scanning by the operator. Alternatively, the display step may involve display on a touchscreen for product selection by the operator.
Preferably, said capturing and storing of the reference image is repeated at intervals.
In another aspect, the invention provides an item recognition apparatus for recognising an individual item from a variety of known products for purposes of entry into an inventory and/or register system, said apparatus comprising:
an identification station having:
a viewplate with backside illumination to provide a backlit image viewable from the frontside of the viewplate;
a digital image capture means arranged to view said backlit image;
a display screen for viewing by an operator; and
an entry means for the operator to selectively make entries into the inventory and/or register system;
computer storage and processing means having:
a first memory store containing data representing a reference image of said viewplate;
a second memory store containing a product database of data sets representing image descriptors characterising features of size and/or shape for all of said variety of known products; and
digitised image processing and analysing means for processing and analysing a digitised image captured by said digital image capture means, including:
means for using the data in said first memory store to subtract the representation of said reference image;
means for generating at least one image descriptor characterising features of size and/or shape of the item to be identified; and
means for comparing said at least one image descriptor with said stored data in said second memory store to identify stored data sets with the highest correlation and to produce a recognition result;
and means for communicating said recognition result to said display screen.
In a preferred form, said entry means is provided by a touchscreen. In an alternative form, the entry means may be a barcode scanner.
The apparatus may include a plurality of identification stations, the computer storage and processing means having a first memory store associated with each identification station and a central second memory store for communication with each of the plurality of identification stations.
Said identification station may include a weighscale associated with said viewplate, said product database including data characterising the weight of all of said variety of known products, and said comparison means including means to compare weight data from an item with stored weight data in the product database.
The invention also provides a checkout station in a retail goods outlet incorporating the above-defined item recognition apparatus.
Also within the contemplation of the invention is a system including the above-defined item recognition apparatus in combination with an inventory and/or register system.